A novel language model based on self-organized learning

Statistical language model is very important to speech recognition. To a system of special topic, domain dependent language model is much better than general model. There are two problems in traditional method to train topic dependent model: 1. The corpus of special topic is not as enough as general corpus. 2. An individual article always relates to more than one topics, traditional method has not considered this phenomena. This paper try to solve these two problems. We have present a new method to organize the corpus--the method based on fuzzy training subset. And the training of domain dependent models are based on these fuzzy subsets. At the same time, a self organized learning approach is introduced in training process to improve the models’ predicting ability. The self organized learning can improve the performance of models evidently. 1. INTRUDUCTION In speech recognition, statistical n-gram model has been successfully used to guild the search and score a path of word string[1]. But a general language model can not use the topic information of speech content efficiently. So the performance of general model will drop when it is used for a specific domain. Topic dependent language model is a effective way to get better performance in special domain. There are two ways to build the topic dependent language model. One is mixture models[2], in this structure, the language models of different topics are all interpolated together according to the mixing factors. It can be expressed as. p w w x p w w i i k k k i i ( | ) ( | ) − − = ∑ 1 1 (1) where xk is the mixing factor of topic k and p w w k i i ( | ) −1 is the language model of topic k. The other is single model structure[3]. In this structure, topic dependent model is interpolated with a general language model which can be expressed as: p w w x p w w x p w w i i g i i k i i ( | ) ( | ) ( ) ( | ) − − − = ∗ + − ∗ 1 1 1 1 (2) where x is weighting factor and p w w g i i ( | ) −1 a general language model. In order to introduce the topic dependent language models into speech recognition, there are two problems must be solved. Firstly, how to update the mixing factors. This problem can be solved by EM algorithm[4], which the mixing factors can be estimated as: x i M x i p w h x j p w h n n i n m n m